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Abstract — This paper presents a Bayesian approach to symbol and phase inference in 
a phase-unsynchronized digital receiver. It primarily extends [lOj to the multi-symbol 
case, using the variational Bayes (VB) approximation to deal with the combinatorial 
complexity of the phase inference in this case. The work provides a fully Bayesian ex- 
tension of the EM-based framework underlying current turbo-synchronization methods, 
since it induces a von Mises prior on the time-invariant phase parmeter. As a result, 
we achieve tractable iterative algorithms with improved robustness in low SNR regimes, 
compared to the current EM-based approaches. As a corollary to our analysis w^e also 
discover the importance of prior regular izat ion in elegantly tackling the significant prob- 
lem of phase ambiguity. 

Keywords — Variational Bayes approximation, phase synchronization, phase ambiguity 
resolution, von Mises distribution, soft decoding 



I Introduction 

Communication Systems face uncertainties of the 
transmitter and receiver oscillators, the noisy dis- 
persive nature of the channel, among others for 
transferring messages from the transmitter to the 
receiver [llj . Incorrect alignment of the transmit- 
ter and receiver oscillators shows itself as a rota- 
tion of the received data on the complex plane and 
renders decoding impossible. The process of align- 
ing these oscillators is known as synchronization 
and is an important aspect in the process of de- 
coding |2]. 

Current state of the art in dealing with this 
problem is a joint iterative estimation of the phase 
and the symbol sequence, in an algorithm known 
as turbo-synchronization [3]. This algorithm is 
based on the framework of an EM-algorithm where 
probability distribution on the symbol decisions 
are used recursively to improve the estimate of 
phase, and vice- versa. This use of probability dis- 
tributions over the possible symbols while making 
a decision, defined as a "soft" decision, was the 
novelty in this approach 3J. The algorithm could 
also be defined under other frameworks, such as 



one based on the sum-product algorithms [1], [7]. 

The EM-algorithm however, considers certainty- 
equivalence measures on the parameter values 
which lead to the loss information about their sta- 
tistical uncertainty. This work aims to general- 
ize this framework into a fully Bayesian treatment, 
which is flexible enough to "learn" the varied chan- 
nel conditions along with the uncertainties asso- 
ciated with the modelled parameters. This was 
however partially achieved in [S], wherein the au- 
thors used a uniform prior over the phase. Good 
Bayesian learning models take into consideration 
all the statistical information, the data has to offer 
and therefore advice better decisions under adverse 
conditions. 

This paper presents the advantage of using a 
generalization of the uniform distribution on the 
unit circle, the Von-Mises distribution in resolv- 
ing phase ambiguity. Literature consists of either 
using a huge number of pilot symbols or modified 
likelihood estimators [4] which are generally ineflfi- 
cient with respect to power and time, to deal with 
the problem. In the course of our analysis, we find 
a natural elegant solution to the problem, which 
does not add any computational burden to the syn- 



chronization routine and exalts the importance of 
prior regularization in Bayesian Analysis. 

As constellation alphabets belong to finite fields, 
the receiver data models are inevitably mixture 
distributions which are not in the exponential fam- 
ily. Finite dimensional sufficient statistics [S] and 
conjugate prior distributions [T] are a unique char- 
acteristic to the exponential family. They are im- 
portant in devising recursive learning algorithms. 
Unfortunately as this is not the case, it leads to 
models with substantially expensive inference pro- 
cedures. A deterministic approximation method, 
known as Variational Bayesian (VB) Method ^^, 
is applied in this paper to deal with the problem. 

The rest of the paper is structured as follows. 
Section II defines the system model encompassing 
unknown phase and symbol sequence. Section III 
describes the single symbol transmission inference. 
Section IV describes the combinatorial complexity, 
the Variational Bayes Method and how it is ap- 
plied to the Multi-Symbol transmission inference. 
In section V, numerical results for the performance 
of the described receiver and its comparisons with 
the current state of the art are provided. Section 
VI discusses the results and its implications. Sec- 
tion VII presents the concluding remarks and fu- 
ture course of actions. 

II System Model 

Structured redundancy is usually added in the 
transmitted data [5], in a process known as cod- 
ing, to protect from the noisy nature AWGN chan- 
nel. Equal partitions of the binary message stream 
formed from uniformly sampled and quantized 
analog signals, are bijectively mapped to a com- 
plex symbol a = [oi, ...a^, ■■■aM]' from a constel- 
lation alphabet A of size M. These symbols are 
then modulated onto a carrier signal which can be 
sent through a physical channel. On the receiver 
side, the reverse operation called demodulation at- 
tempts to decode or identify the transmitted sym- 
bol sequence, which are now corrupted by noise 

We observe the data vector Xi G C" dur- 
ing every symbol period with i e 1,...K with 
Xi — [x'i,X2, ...x'^' and assume the prior proba- 
bilities corresponding to the symbol vector a to 
be a = [ai, ...om]'- </* represents the phase dif- 
ference of the received signal to the transmitted, 
with noise r in an AWGN channel [TU]. Ease 
of inferential analysis dictates the augmentation 
of the model using a latent variable. Let it be 
h = [Ii,i,---Im,i]' e {eM(l),---ej\/(M)}, where 
^AiiiT^) = [S{m— 1), ...S{m — M)]' and S is the Kro- 
necker Delta. Hence k is a vector pointer to the 
component active at the z"* symbol period. The 
data generation model for such a symbol transfer 



in a single symbol period can be defined as: 

f{x^\(|),r,a) = ^f{k\a)f{xi\a,(j),r,k) (1) 



f{li\a) = Mu^ (l,a) = l[a 



(2) 
(3) 



where CM: Complex Normal and M.u: Multi- 
nomial distributions, 4> g [— tt, tt], r G (0, oo), 
g^ = [sie^'^,...,s„e-''^"]' with s, £ {l,2...n} as the 
pulse sequence and lo the carrier digital frequency 
which are assumed to be known. In this analysis 
we assume known SNR, based on which we can 
calculate the noise variance, r. 

a) Prior Distribution to Phase: Von-Mises Dis- 
tribution 

As we will see later, the VB approximation on the 
joint distributions lead to marginal distributions of 
the phase from the exponential family, hence the 
use of conjugate priors remains important. Von- 
Mises distribution [12J, [8J is a useful distribution 
over the unit circle defined from [— tt, tt] on the real 
line, which serves as a conjugate prior to the phase 
for our conditional data model. It has one com- 
plex parameter, which serves as both location and 
concentration parameter for the distribution and 
has been used in varied applications on directional 
data. 

/(0|ko) = M{ko) ^ -—— rr 

2tt1o[\i^o\) 



,(5R{«oe-^*}) (4) 



where kq G C. In the signal processing literature 
this distribution is also known as the Tikhonov 
distribution [12] and has been previously used to 
model the phase error from the estimated phase us- 
ing a phase- locked loop (PLL). In this work Von- 
Mises/Tikhonov distribution is considered to be 
the prior distribution of the phase in a Bayesian 
framework which was explicitly introduced in |10j . 
As the Von-Mises/Tikhonov distribution is con- 
fined to the unit circle, means and variances out- 
side [— 7r,7r] are undefined. Hence circular mo- 
ments are used which are defined as: 



E[0] = Zko 
Varcir [< 



l-E[cos(0-E[(/>])] 
^id^ol) 



1 



^(l^ol) 



(5) 



(6) 



Limiting behaviour of this distribution under vari- 
ous conditions of its hyperparameter are as follows 



lim /(0|ko) = A/'0 ( Zko, - — r ) 
lim f{(f)\Ko) =U^{[~Tr,TT]) 

|ko|->-0 



(7) 
(8) 



where Af is the normal distribution and U is the 
uniform distribution. 



Ill Single Symbol Transmission 

Now we present the inference procedure for the 
single symbol transmission, where exact inference 
was shown to be possible in [10]. This analy- 
sis forms the basis for our generalization into the 
multi-symbol cases, wherein the Variational Bayes 
Method is applied. The data generation model in 
this case is taken to be 



fixJ,(t)\g,Ko,a,l3o,a) 



where 



f{x\a,(l),r,l)f{l\a)f{(t)\Ko) 
(9) 



f{(b\Ko) = MiKo) 

f{l\a) = Mui^ (1, a) = I' a 
f{x\a,<l>,r,l)=CAf^{l'age^tr) 



(10) 

(11) 
(12) 



The marginal posterior distributions for phase 




Fig. 1: Graphical Model - Single-symbol model in the 
case of uncertain phase 



and the symbol can be easily calculated in this 
case and are given in [TD] to be 



I 
f{l\^)^Muiil,pi) 



where 



Ki = Ko + -{l'ag)"yi 



Ko 



{l'a)"Y: 



^k^k^ 



-juk 



fc=l 



PiCx(/'a)e(-^('''^«)"('''^«))/o(ki|) 



(13) 
(14) 

(15) 
(16) 
(17) 



The mean value of (j) for each component is given 
by Zki. Also ki can be characterized as a Distrete 
Time Fourier Transform on the incoming data, 
which is intuitive since ki is a sufficient statistic 
to the phase (p. Here however we get more infor- 
mation than from a simple DTFT, namely about 
the confidence over the expected phase. 

Another important observation about ki can be 



seen with the following algebraic expansion: 



\Ki 



[HlKll 



[^{l'ag)"^{l'ag)"^ 



:-^(l'nn-)H^]l/2 



+ kqKq + -Ko(l'ag)^x+ -Ko(l'ag) xl 
r r 



(19) 



From the above equation we can see that when 
kq = (Uniform Prior), all except the first term 
vanishes. That implies then that when the prior is 
a uniform distribution, the decoding distribution 
is invariant to the symbols on the same radius in 
a QAM constellation, which is the origin of phase 
ambiguity. However a non-zero kq makes ki de- 
pend on the value of the symbols and hence results 
in a unique MAP from the decoding distribution. 

IV Multi-Symbol Transmission 

Real world communication systems involve mul- 
tiple symbol periods, which are observed sequen- 
tially and a decoding distribution for the symbols 
is the desired output. The joint data model under 
the independent symbol sequence assumption (rea- 
sonable in turbo processing where pseudorandom 
inter-leavers are typically employed p]) is then 

K 

/(x^|0, r, a) = n E /('«l«)/(^d«' -^^ r^ Q (20) 

As a consequence of this product over a sum- 
mation, M^ term mixture model appears after 
K time periods, a combinatorial explosion which 
leads to intractable exact inference. 

a) Variational Bayes Method 

A deterministic approximation, known as the Vari- 
ational Bayes Method is employed to deal with this 
computational intractability. The Method aims 
at minimizing the KuUback-Leibler Divergence be- 
tween the candidate approximation and the origi- 
nal joint distribution. This divergence can be in- 
terpreted as an information difference |15j between 
the two distributions. The intuition then is to get a 
functional form of the posterior marginals, so as to 
minimize the information lost due to the forced in- 
dependence [13] . These methods have found many 
applications in machine learning, artificial intelli- 
gence and more recently in signal processing [S]. 
The essential result which forms the basis of this 
method is as follows: 

Theorem IV. 1 Variational Bayes Method IJJJ: 
Let f{9\x) be the posterior distribution of multi- 
variate parameter, 9. The latter is partitioned into 
q sub-vectors of parameters or q-nodes: 



9 = [9'i,9'2, ...9']' 



(21) 



Let f{0\x) he an approximate distribution re- 
stricted to the set of conditionally independent dis- 
tributions for 61,62, ■■■Oq: 



he\x) = \{f{6^x) 



(22) 



Then, the minimum of Kullhack-Leibler divergence 
KL (J{6\x)\\f{6\x)\ is reached for 



/(0,|a;)cxeV^^-/ 



(«,.|.)[ln(/(e^^))] 



(23) 



where i ~ \,...,q and 6u in 6, and f{6u\x) = 

X\%x,J^^KQM)■ Now ~f{0\x) *5 the VB- 

aproximation, and f(6i\x) are the VB-marginals. 

b) Offline VB Approximation 

Consider the situation, where we observe a batch of 
symbol transfer periods together and are required 
to infer the marginal distributions for all the trans- 
ferred symbols and the phase parameter. Prior in- 
dependence of the symbols is assumed. However 
it should be noted, that aposteriori the symbols 
are independent conditional on (p. Expanding the 
joint probability, over all unknown parameters, we 
have 

/(xA',lA:,0|Ko,a,r) 

K 

= /(0l«o) n /(^>' '^' ^' ^0/('*l«) (24) 



4=1 



where 



/(</.|«o) = X(«:o) (25) 

f{k\a)=Mui^{l,a)=l[a (26) 

/(x,|a, 0, r, k) = CN^^ {l[ag,e=^, r) (27) 

From the above joint distribution, we find the 




Fig. 2: Graphical Model - Multi-symbol model in the 
case of uncertain phase 

posterior distributions for the label field and the 
phase. Integrating out the label field entails a M^ 
summation, making it intractable |13j . We there- 
fore resort to using Theorem II. 1 and derive the 



approximate posterior distributions of the param- 
eters as 

/(0|xk) oc e(%.^|.K)['"/("^''^-'^l««'"''-)l) 

= M^{ki^) (28) 



(29) 
with the shaping parameters as 



f(k\y.K) oc e 

^ Mui.{l,p^ 



K 



Kl 



Kq 



^{i[agi)"xi 

i=l 

II' \ (--{'■',ag,)"(l',ag,)+^5A 



(30) 
(31) 



and the associated VB moment as 

1^ J -'od'ti^lj 



(32) 



Ki^ again can be characterized as a DTFT. Impor- 
tant subsets of this method discussed in the litera- 
ture include [9] where the authors have assumed an 
uniform prior to the phase which implies kq = 0. 
Further if we make the ratio 



A(l^i^l) 
-fo(lKii^l) 



= 1 



we get the EM-algorithm based turbo- 
synchronization routine as discussed in j3]. 

c) Online VB Approximation 

In this case we observe each symbol transfer pe- 
riod sequentially and are required to infer the 
marginal distributions for each transferred sym- 
bol and phase parameter, after each symbol pe- 
riod. Again prior independence of the symbols is 
assumed. To deal with the increasing combina- 
torial complexity after each time step we conflate 
the posterior distribution of (j) which is a mixture of 
M Von-Mises components using Variational Bayes 
Method in to a single Von-Mises posterior. The as- 
sociated sufficient statistic can then be used as the 
prior for the next symbol period. Expanding the 
joint probability, over unknown parameters after 
each time step, we have 

f{xt,lt,(l>\^t-i) = f{xt\a,(j>,r,lt)f{lt\a)f{(l)\xt-i) 

(33) 



where 

/(0|xo)=/(0|^o) = X('«o) 
fikla) = Mui^ {l.a) = l[a 

fixt\a,<f>,r,k)=CN:.,{l[agte^'^,r) 



(34) 
(35) 
(36) 



On equation (13) from Section III of the posterior 
distribution of the phase, we apply the VB Method 
and derive the approximate conflated posterior dis- 
tribution as 

/((/>|Xt_i) OC g(%«t-i|.t_.i)[l"/(^*-i''*-i''^l''*-2.'=o."^'-)l) 

= X^K_J (37) 

(38) 

with the shaping parameter as 

Kit-i = Kit-2 + -{I't-iagt^i)" xt^i (39) 

pi,_, (X (Cia)e(-^(''-'^«)"('-^"^))/o(k,,_J) 



k-i = E\lt-i] 



(40) 
(41) 



Using this shaping parameter as initiaUzation for 
the following symbol period we get from the single 
symbol transmission case: 



/(0|x,) = 5]p,,X(mJ 

/(?t|xt) =Mui,{l,piJ 

where 

2 



Ki, = Ky 



_, + -{l[agt)"xt 



Pi, ex (/:a)e(-^('''"3)"('-«^))/o(|^iJ) 



(42) 
(43) 

(44) 
(45) 



Kij again can be characterized as a DTFT as ear- 
lier. 

V Simulation 
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Fig. 3: - Performance of the VB-based decoding 
algorithms in the multi-symbol case 

The above figure describes the proportion of suc- 
cessful identifications of the symbol transferred for 
various values of SNR, using the methods devel- 
oped in this paper. In each batch 20 symbols are 
transmitted through an AWGN channel. In the 
low-SNR regime, from -15dB to 5dB we see that 
the Bayesian algorithms are significantly better. 
Independent decoding algorithm uses the exact de- 
coding algorithm over each subsequent symbol pe- 
riod ignoring the previous periods. This method 



turns out to be the fastest and most accurate for 
this simulation. The Online VB and the Offline 
VB approximations provide almost as good results, 
though they would be more robust in a more gen- 
eral setting. We can also clearly see the perfor- 
mance of |H] and [T is significantly lower with re- 
spect to the decoding success. It should be noted, 
to deal with the problem of phase ambiguity for 
these two methods, 5 pilot symbols are used in 
front of the original 20. 

VI Discussion 

We get two main results from the analysis per- 
formed in this paper. Primarily we are able to 
extend the EM-Algorithm based synchronization 
into a fully Bayesian synchronization procedure. 
This can form the basis for future generalizations 
and analysis of even more complicated scenarios. 
Secondly, as a corollary to our analysis we are able 
to resolve the problem of phase ambiguity. 

As shown in the Section III, setting kq = ren- 
ders Ki independent of the expected phase £[(/)], 
which thereby results in the apparent ambiguity 
in the decoding distribution. The notion of non- 
zero Kq seems quite inconsequential, but a concen- 
trated prior however informs the presence of a uni- 
modal phase distribution and regularizes the ob- 
servation model which is intrinsically rotationally 
invariant for QAM. It must be noted however, that 
as the number of incoming observations increases 
the sampling distribution of the MAP symbol be- 
comes relatively invariant to the value of kq, as 
long as Ko > 0. 

Computationally it presents no overhead, as it 
involves only the addition of a constant. In con- 
trast past approaches like Differential Coding [2], 
angle differential QAM scheme [6] and the use of 
pilot symbols [3] [1] , among others which have been 
proposed to deal with this rotational invariance are 
visibly more complex and lack the simplicity of this 
observation. 

In [3] the authors use the EM-Algorithm to deal 
with phase synchronization. This essentially im- 
plies using the sifting property of the Kronecker 
Delta function and thus using the expectations of 
the parameters in the associated functions rather 
than the expectations of the functions of the pa- 
rameters itself. Significantly lower accuracy of the 
decoding distribution in the low-SNR regime is ob- 
served which converges in performance with in- 
creasing SNR. 

VII Conclusions 

In this paper we have been able to extend the EM 
framework for tackling the phase-synchronization 
problem, to a fully Bayesian VB based algorithm 
wherein as a virtue of its modelling ability the 
problem of phase ambiguity does not arise. This 



makes the concept of using "soft" information 
truly complete as no statistical information is lost 
at any point in the iterations. Further the ba- 
sic framework developed motivates further relax- 
ations into more general channels, such as those 
with multi-path fading, like Rician Fading [T. In 
future work we would also be relaxing the condi- 
tion on known noise parameter and deal with it in 
a Bayesian framework. 
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